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(g) Image encoding apparatus. 

(57) In an image encoding apparatus, image infor- 
mation is quantized and variable-length encod- 
ing is performed thereon so that the data are 
stored in a buffer. Thereafter, the data in the 
buffer are transmitted to the outside at a fixed 
data rate. At this time, a controller monitors a 
number of occupied bits in the buffer, and when 
the number of occupied bits is not more than 
thresholds, the controller switches the output of 
the apparatus to a dummy data generator. At 
this time, a parameter which is used for rate 
control is initialized, and dummy data which 
can be eliminated on a decoding circuit are 
outputted from the dummy data generator. Fur- 
thermore, bit allocation is changed so that a 
sum of the number of occupied bits and the bit 
allocation to a frame which is determined bef- 
ore encoding does not exceed a fixed amount 
Moreover, when the number of occupied bits 
exceeds the fixed value, a quantization step size 
is changed to a greater value. This efficiently 
prevents underflow . and overflow of the buffer, 
and the image information can be encoded with 
excellent quality. 
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FIELD OF THE INVENTION 

The present invention relates to an image encod- 
ing apparatus which is provided to an apparatus for 
performing a digital image process, such as a video- 
phone, a digital video disk apparatus, and which en- 
codes image information efficiently in order to im- 
prove efficiency of recording and transmitting of im- 
age information. 

BACKGROUND OF THE INVENTION 

In the case where image information with a wide 
band is recorded or transmitted, if the image informa- 
tion have its original configuration, a capacity of a re- 
cording medium and a transmitting medium is re- 
quired for an amount of its data amount, so efficiency 
of recording and transmitting is deteriorated. For this 
reason, an amount of data of image information is 
generally compressed so as to be suitable for an abil- 
ity in recording of a recording medium or an ability in 
transmitting of a transmitting medium. Such a data 
compressing process is realized by encoding high- 
efficiently digital motion picture information by 
means of arts, such as motion compensation inter- 
frame prediction, orthogonal transformation, quanti- 
zation, variable-length encoding. 

For example, in a high-efficient encoding system 
cited in the literature "MPEG2 Interframe Prediction" 
(the Institute of Television Engineers, Technical Re- 
port Vol. 16 No. 61 . pp. 37-42), there exist a frame on 
which only intra-frame encoding is performed (here- 
inafter, referred to as I picture), a frame on which pre- 
dictive encoding from the previous frame is per- 
formed (hereinafter, referred to as P picture) and a 
frame on which predictive encoding from the previous 
and subsequent frames is performed (hereinafter, re- 
ferred to as B picture). An amount of codes generated 
in I picture is great, an amount of codes generated in 
P picture is smaller than that in the I picture, and an 
amount of codes generated in B picture is fairly small- 
er than those in the I picture and the P picture. 

When the high efficient encoding system is ap- 
plied to image transmission and image storing (re- 
cording), there arises a problem how a data rate after 
encoding (hereinafter, referred to as encoded data) is 
controlled. 

For example, the literature "MPEG2 Quantization 
and Coding Control (the Institute of Television Engi- 
neers, Technical Report Vol. 16 No. 61. pp. 43-48) dis- 
closes a method for almost constantly controlling an 
average number of bits in a given interval which is 
called group of pictures (hereinafter, referred to as 
GOP) (an interval from I picture to next I picture, and 
a number of pictures in the GOP is normally ten-odd 
frames). 

Fig. 16 is a block diagram of an image encoding 
apparatus utilizing a conventional method. 



First, in a block dividing circuit 51, a frame to be 
encoded is divided into blocks which are called a mac- 
roblock (hereinafter, referred to as MB). Intra-frame 
encoding or inter-frame encoding is performed per 
5 MB. 

In a motion compensation inter-frame predicting 
circuit 52, a current frame is compared with a frame 
previously encoded and decoded so that while an 
amount of a move of MB (move vector) is being de- 

10 tected, the MB is predicted from a decoded frame. 
However, MB of I picture and MB in which intra-frame 
encoding is advantageous are not predicted. The pre- 
dicting method is disclosed in the literature "MPEG2 
Interframe Prediction" (the Institute of Television En- 

15 gineers, Technical Report Vol. 16 No. 61. pp. 37-42). 

In an orthogonal transforming circuit 53, the data 
of MB are transformed to data suitable for encoding 
by two-dimensional orthogonal transformation (for 
example, discrete Fourier transformation, Hadamard 

20 transformation, discrete cosine transformation, etc.). 
At this time, in the case of intra-frame encoding, two- 
dimensional orthogonal transformation is carried out 
on the data of MB from the block dividing circuit 51, 
whereas in the case of inter-frame encoding, a switch 

25 54 is switched to the motion compensation inter-fra- 
me predicting circuit 52, a prediction difference (pre- 
diction error) is obtained by a subtracter 55 and two 
-dimensionally orthogonal transformation is carried 
out on the prediction error. 

30 In a quantizing circuit 56, transform coefficients 

are quantized by a quantization step size obtained in 
a quantization step size determining circuit 66, and in 
a variable-length encoding circuit 57, variable-length 
encoding is performed on the quantized transform 

35 coefficient. Then, in a buffer 58, the output from the 
variable-length encoding circuit 57 is temporarily stor- 
ed and encoded data are outputted at a determined 
constant data rate. 

In addition, the transform coefficient quantized in 

40 the quantizing circuit 56 are inversely quantized in an 
inverse quantizing circuit 59 and inverse orthogonal 
transformation is carried out on inversely quantized 
transform coefficient in an inverse orthogonal trans- 
form circuit 60. 

45 In the case of inter-frame encoding, since the re- 

constructed value from the inverse orthogonal trans- 
forming circuit 60 is the data of the prediction differ- 
ence, a switch 62 is switched to a terminal 62a side, 
and in an adder 61 , the data of the prediction differ- 
so ence as the reconstructed value and prediction MB 
outputted from the motion compensation inter-frame 
predicting circuit 52 are added so that reconstructed 
image data can be obtained. Meanwhile, in the case 
where the reconstructed value from the inverse or- 

55 thogonal transforming circuit 60 is the data of MB on 
which intra-frame encoding is performed, since the 
reconstructed data directly becomes reconstructed 
image data, the switch 62 is switched to a terminal 
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62b side. 

The reconstructed image data are stored in a 
frame memory 63. The reconstructed image data are 
used as a reference image for inter-frame prediction 
in the motion compensation inter-frame predicting cir- 
cuit 52. 

In addition, in the conventional image encoding 
apparatus, encoding control is performed by a bit al- 
locating circuit 64, a reference quantization step size 
determining circuit 65 and a quantization step size de- 
termining circuit 66. The following will discuss the en- 
coding control in detail. 

The bit allocating circuit 64 determines a number 
of bits which are allocated to a frame to be encoded 
per frame based upon the encoded results up to the 
previous frame (a number of generated bits obtained 
from the various-length encoding circuit 57, and an 
average value of quantization widths obtained from 
the quantization step size determining circuit 66) and 
the given data rate. Concretely, global complexity 
measures X,, X P and Xe of I picture, P picture and B 
picture are determined according to the following for- 
mulas (1) through (3). 

X, = S, Q, (1) 
X P = SpQp (2) 
X B = S B Q B (3) 
In the formulas (1) through (3), S (f S P and S B rep- 
resent an amount of encoded data of previous I pic- 
ture, P picture and Bpictureja number of generated 
bits). Moreover, Q|, Q P and Q B represent an average 
quantization step size at the time of encoding of pre- 
vious I picture, P picture and B picture. 

The global complexity measures are based upon 
that when an image to be encoded is complex, a num- 
ber of generated bits increase and a quantization step 
size tends to be enlarged in order to restrict increase 
in the number of generated bits. However, a fixed ini- 
tial value is set as the global complexity measures at 
the beginning of encoding. 

Next, as to the bit allocating circuit 64 determines 
target bit allocation T,, T P and T B to an image to be en- 
coded of I picture, P picture and B picture according 
to calculation of the following formulas (4) through (6). 
R 



T, = 



+ Np X P N B X B 
Kp Xj K B Xj 



(4) 



t P = 



N D + 



N B K P X B 
Ke Xp 



T B = 



N B + 



NpKsXp 



(5) 



(6) 



KpX B 

In the above formulas (4) through (6), R repre- 
sents a number of remained bits in GOP, N P and N B 
represent a number of P pictures and B pictures 
which have not been encoded yet in GOP, and Kp and 
K B represent constants. 



Next, the reference quantization step size deter- 
mining circuit 65 and the quantization step size deter- 
mining circuit 66 determine a quantization step size 
by using the values T,, T P and T B . In the case of I pic- 
5 tures, in order to obtain a quantization step size, a ref- 
erence quantization step size which becomes a ba- 
sis is first obtained and is compensated according to 
an activity of an image so that a quantization step size 
Qj' (j is a number of MB) is obtained. Also as to P pic- 
w ture and B picture, since a quantization step size can 
be determined in the same manner as of I picture, the 
following will discuss determination of a quantization 
step size of I picture. 

First, in the reference quantization step size de- 
ls termining circuit 65, a reference quantization step 
size of orthogonal transformation coefficient is deter- 
mined per MC based upon a number of generated bits 
up to the previous MB obtained in the variable-length 
encoding circuit 57 and a number of bits T, which is 
20 allocated by the bit allocating circuit 64. 

In other words, the reference quantization step 
size determining circuit 65 determines virtual buffer 
fullness d/ in the j-th block of I picture according to the 
following formula (7). 

25 

(7) 



d| = d' + B. , - Mill 
1 ° j 1 MBcnt 



The reference quantization step size determining 
circuit 65 determines a reference quantization step 
size qj' per MC according to calculation of the follow- 
30 ing formula (8). 



(8) 



In the formula (7), d 0 ' is an initial buffer fullness 
(final buffer fullness q MBcnt I at the time of encoding 

35 previous I picture, and its initial value is a certain con- 
stant), Bj is a total number of bits generated in an im- 
age up to the j-th block, and MBcnt is a total number 
of MB in an image. Furthermore, r in the formula (8) 
is a constant which is called a reaction parameter. 

40 In the formula (7), T, bit is equally allocated to 

each MB, and a difference between a number of al- 
located bits up to the previous MB and a number of 
actually generated bits is reflected in the virtual buf- 
fer fullness. Therefore, if a number of generated bits 

45 up to the previous MB is great, the virtual buffer full- 
ness becomes large, and also the reference quanti- 
zation step size becomes large, so a number of sub- 
sequently generated bits is restricted. On the contra- 
ry, if the number of generated bits up to the previous 

50 MB is small, the virtual buffer fullness becomes 
small, and also the reference quantization step size 
becomes small, so a number of subsequently gener- 
ated bits is enlarged. Here, in P picture and B picture, 
reference quantization widths q/ 5 and qj B are deter- 

55 mined in the same manner. 

In the quantization step size determining circuit 
66, an activity which shows fineness of a picture in 
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MB obtained from the block dividing circuit 51 is cal- 
culated per MB, and the reference quantization step 
size q/ is changed by the activity per MB so that a 
quantization step size which is used in the quantizing 
circuit 56 is determined. 

In other words, the quantization step size deter- 
mining circuit 66 determines the quantization step 
size Q/ of the j-th MB in I picture is determined accord- 
ing to the following formula (9). 

Q| = qj Nactj (9) 

Here, Nactj in the formula (9) is an activity of the 
j-th MB which is normalized, and it is represented by 
the following formula (10). 

Nac^ = 2i f V" 9 (10) 
1 actj + 2avg 

In the formula (10), actj is the activity of the j-th 
MB, and avg is an average value of the activities in a 
previously encoded frame (it is not always I picture) 
(however, a predetermined initial value is set for ac- 
tivity at the beginning of encoding). Here, also in P 
picture and B picture, quantization widths Qj p and Qj B 
are determined in the same manner as in the above. 

In order to explain the problem of the convention- 
al image encoding apparatus, the following will dis- 
cuss a number of occupied bits in the buffer 58 in the 
image encoding apparatus (different from the virtual 
buffer fullness mentioned in the prior arts). 

Here, the occupied bits in the buffer 58 is defined 
as follows. In other words, in the case where after en- 
coding of each image frame and storing of encoded 
data in the buffer 58 are instantly completed in an im- 
age encoding apparatus, the encoded data are out- 
putted at a constant data rate, buffer occupied bits 
are defined as a number of bits of the encoded data 
which remain in the buffer 58. 

Fig. 17 shows an example of an aged change in 
the number of occupied bits in the buffer 58 with re- 
spect to a typical motion picture sequence (image 1). 
Here, a horizontal axis shows a time which is repre- 
sented by a frame, a vertical axis shows the number 
of occupied bits in the buffer 58, and an initial number 
of occupied bits is 0. Moreover, a bit rate is 6 Mbps, 
a number of frames in GOP is 15, and an interval be- 
tween I picture and P picture and an interval between 
P pictures are 3 frames. 

As is clear from Fig. 17, at time 0 a lot of data are 
generated by encoding I picture, but as to subsequent 
P picture and B picture, an encoding amount is adjust- 
ed, so the number of occupied bits in the buffer 58 
substantially returns to the initial value with respect to 
first images in each GOP (time 15, 30, 45, etc.). In this 
manner, if data amount control in the GOP is satisfac- 
torily made, the first number of occupied bits in the 
buffer 58 in each GOP always returns to near the ini- 
tial value. Therefore, it is sufficient that a capacity of 
the buffer 58 is 10 6 bit, for example. 

The following will discuss a motion picture se- 



quence including stop motion or frequent scene 
changes. In a sequence which is shifted to a normal 
motion picture after stop motion continues for 30 
frames (image 2), a number of occupied bits in the 

5 buffer 58 is shown by a solid line in Fig. 18. In a se- 
quence which is shifted to a normal motion picture af- 
ter frequent scene changes continue for 30 frames 
(image 3), a number of occupied bits in the buffer 58 
is shown by a broken line in Fig. 18. 

10 In the case where the capacity of the buffer 58 is 

1 0 6 bit, as shown in fig. 18, data overflow greatly after 
time 15 and after time 30 in the image 2, that is, over- 
flow occurs. The overflow occurs after time 15 of the 
image 2, for when a new GOP starts in a stop motion 

15 and I picture appears, extraordinarily many number of 
bits are allocated to the I picture and a lot of encoded 
data is generated. Moreover, also in I picture in a mo- 
tion picture immediately after the stop motion, a lot of 
encoded data is generated, so overflow occurs even 

20 after time 30 in the image 2. 

In order to avoid such overflow, the capacity of 
the buffer 58 should be enlarged, but this arises a 
problem that a scale of hardware increases. More- 
over, in the case where a lot of encoded data is gen- 

25 erated partially (all at once) compared to the above- 
mentioned normal sequence, a remaining number of 
bits allocated to GOP (a number of remaining bits in 
GOP) is extremely decreased. Therefore, there aris- 
es a problem that quality of an image thereafter is re- 

30 markably deteriorated. 

In addition, there exist no encoded data in the 
buffer 58 after time 53 of the image 2, at times 25 
through 29 of the image 3, etc., so underflow occurs. 
This can be a problem in actual hardware. Moreover, 

35 in the case where underflow occurs, a rate is not suc- 
cessively controlled, so there arises a problem that 
quality of an image is remarkably deteriorated due to 
encoding by using an inappropriate quantization step 
size. 

40 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide 
an image encoding apparatus which is capable of en- 
45 coding an image with good quality and without under- 
flow of a buffer. 

In order to achieve the above object, the image 
encoding apparatus of the present invention is char- 
acterized by including: 
so (1) quantizing means for quantizing information 

which corresponds to an image; 
(2) variable-length encoding means for perform- 
ing variable-length encoding on the quantized re- 
sult of the quantizing means; 
55 (3) temporary storage means for temporarily stor- 

ing encoded data from the variable-length encod- 
ing means so as to transmit the stored encoded 
data to the outside at a fixed data rate; 
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(4) encoded data amount control means for cal- 
culating a quantization step size for quantizing in- 
formation to be encoded based upon the encod- 
ed result of the variable-length encoding means 
so as to control an amount of the encoded data 
of the variable-length encoding means; 

(5) dummy data generating means for generating 
dummy data which can be eliminated on a decod- 
ing side for decoding the encoded data; 

(6) switching means for switching an output of the 
image encoding apparatus to either the encoded 
data from the temporary storage means or to the 
dummy data from the dummy data generating 
means; and 

(7) control means for monitoring a number of oc- 
cupied bits in the temporary storage means so as 
to control a switching operation of the switching 
means so that the output of the image encoding 
apparatus becomes the encoded data from the 
temporary storage means when the number of 
occupied bits exceeds a predetermined thresh- 
old, whereas so as to suspend transmission of 
data in the temporary storage means and control 
the switching operation of the switching means so 
that the output of the image encoding apparatus 
becomes the dummy data generated from the 
dummy data generating means when the number 
of occupied bits is not more than the threshold. 
With the above arrangement, the information cor- 
responding to image information is quantized by the 
quantizing means and variable-length encoding is 
performed on the quantized result by the variable- 
length encoding means so that the encoded data are 
stored in the temporary storage means. In this case, 
an amount of encoded data in the variable-length en- 
coding means is controlled by the encoded data 
amount control means which calculates a quantiza- 
tion step size of information to be encoded based 
upon the encoded result of the variable-length encod- 
ing means so as to adjust a quantization step size of 
quantization in the quantizing means according to the 
calculated result. The encoded data stored in the tem- 
porary storage means are transmitted to the outside 
at a data rate, which is suitable for a process by an 
outer apparatus. At this time, it is necessary to adjust 
a number of occupied bits in the temporary storage 
means (an amount of encoded data stored in the tem- 
porary storage means) so that the generated encod- 
ed data do not exceed a memory capacity of the tem- 
porary storage means (occurrence of overflow), and 
the encoded data to be transmitted to the outside do 
not run out in the temporary storage means (occur- 
rence of underflow). 

Therefore, the dummy data generating means for 
generating dummy data which can be eliminated on 
a decoding circuit for decoding the encoded data is 
provided so that an output of the image encoding ap- 
paratus can be switched to either the encoded data 



from the temporary storage means or the dummy 
data from the dummy data generating means by the 
switching means. Furthermore, the number of occu- 
pied bits in the temporary storage means is monitored 
5 by the control means. The switching operation of the 
switching means is controlled by the control means 
so that if the number of occupied bits exceeds the 
predetermined thresholds, the output of the image 
encoding apparatus is the encoded data from the 

10 temporary storage means, and if the number of occu- 
pied bits is not more than the thresholds, the output 
of the image encoding apparatus is the dummy data 
from the dummy data generating means. 

Therefore, if the number of occupied bits in the 

15 temporary storage means is not more than the thresh- 
olds, the dummy data from the dummy data generat- 
ing means are outputted from the image encoding ap- 
paratus instead of the encoded data read out from the 
temporary storage means. While the dummy data are 

20 being outputted, the transmission of the encoded 
data in the temporary storage means is suspended. 
During this operation, if the number of occupied bits 
in the temporary storage means exceeds the thresh- 
olds, the encoded data in the temporary storage 

25 means are again outputted at a constant data rate. 
For this reason, a lower limit of the number of occu- 
pied bits in the temporary storage means can be sub- 
stantially limited to the thresholds, so the underflow 
of the temporary storage means can be prevented. 

30 In addition, it is preferable that when the number 

of occupied bits is not more than the thresholds, the 
control means (7) outputs a switching signal to the en- 
coded data amount control means, and when the 
switching signal is inputted, the encoded data amount 

35 control means (4) controls initializes various parame- 
ters which are used for calculation of the quantization 
step size. As a result, even when the dummy data are 
outputted instead of the actual encoded data, the 
parameters which are used for rate control can keep 

40 their balance and the stable rate control can be made. 
Therefore, a deterioration in quality of an image can 
be restrained. 

It is another object of the present invention to pro- 
vide an image encoding apparatus which is capable 

45 of encoding an image without overflow in a buffer and 
with good quality. 

In order to achieve the above object, another im- 
age encoding apparatus of the present invention hav- 
ing the quantization means (1), the variable-length ■ 

so encoding means (2) and the temporary storage 
means (3) is characterized by including: 

(8) encoded data amount control means for 
calculating bit allocation per frame of information to 
be encoded based upon the encoded result of said va- 

55 riable-iength encoding means and calculating a quan- 
tization step size from the above calculated result so 
as to control an amount of the encoded data of said 
variable-length encoding means by adjusting the 
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quantization step ?. «5 of quantization in said quantiz- 
ing means accord r to the above calculated result, 
wherein sa encoded data amount control 
means (8) includes bit allocation changing means for 
monitoring a number of occupied bits in the tempor- 
ary storage means so as to change the calculated bit 
allocation when the sum of bit allocation to a frame 
which has been calculated before encoding and a 
number of occupied bits in said temporary storage 
means at the time of the calculation of the bit alloca- 
tion exceeds the predetermined thresholds, such that 
the sum does not exceed the thresholds. 

With the above arrangement, the encoded data 
amount control means calculates bit allocation per 
frame of information to be encoded so as to determine 
a quantization step size. The encoded data amount 
control means includes bit allocation changing means 
for changing the calculated bit allocation. The bit al- 
location changing means monitors the number cf oc- 
cupied bits in the temporary storage means and 
changes the calculated bit allocation so that a sum of 
the bit allocation to a frame calculated before encod- 
ing and the number of occupied bits in the temporary 
storage means at the time of calculation of the bit al- 
location do not exceed the predetermined thresholds. 
Therefore, even when the frame information are en- 
coded and stored in the temporary storage means, 
the number of occupied bits in the temporary storage 
means at this time do not greatly exceed the thresh- 
olds. As a result, the overflow can be prevented by 
setting the thresholds to smaller values than the 
memory capacity of the temporary storage means. 

In addition, it is desirable that the encoded data 
amount control means (8) further includes quantiza- 
tion step size changing means for monitoring the 
number of occupied bits in the temporary storage 
means so as to change the calculated quantization 
step size to a greater value when the number of oc- 
cupied bits exceeds the f ixed thresholds. This makes 
it possible to efficiently prevent the overflow of the 
temporary storage means. 

For a fuller understanding of the nature and ad- 
vantages of the invention, reference should be made 
to the ensuing detailed description taken in conjunc- 
tion with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram which shows an arrange- 
ment of a main circuit in an image encoding apparatus 
of the first embodiment of the present invention. 

Fig. 2 is a graph which shows an aged change in 
a number of occupied bits in a buffer in the case 
where information corresponding to an image (image 
3) which is shifted to a normal motion picture after 
frequent scene changes continue for 30 frames is en- 
coded by the image encoding apparatus shown in 
Fig. 1. 



Fig. 3 is an explanatory drawing which explains 
an arrangement of GOP and a inter-frame predicting 
method. 

Fig. 4 is a block diagram which shows an arrange- 
5 ment of a main circuit in an image encoding apparatus 
of the second embodiment of the present invention. 

Fig. 5 is a graph which shows an aged change in 
a number of occupied bits in a buffer in the case 
where information corresponding to the image 3 is en- 
w coded by the image encoding apparatus shown in 
Fig. 4. 

Fig. 6 is a block diagram which shows an arrange- 
ment of a main circuit in an image encoding apparatus 
of the third embodiment of the present invention. 

15 Fig. 7 is a graph which shows an aged change in 

a number of occupied bits in a buffer in the case 
where information corresponding to an image (image 
2) which is shifted to a normal motion picture after 
stop motion continues for 30 frames is encoded by 

20 the image encoding apparatus shown in Fig. 6. 

Fig. 8 is a block diagram which shows an arrange- 
ment of a main circuit in an image encoding apparatus 
of the fourth embodiment of the present invention. 
Fig. 9 is a graph which shows a relationship be- 

25 tween a value a, which is multiplied by a quantization 
step size so as to change the quantization step size 
used for the image encoding apparatus shown in Fig. 
8, and a number of occupied bits in a buffer BO. 
Fig. 10 is a graph which shows an aged change 

30 in a number of occupied bits in a buffer in the case 
where information corresponding to the image 2 is en- 
coded by the image encoding apparatus shown in 
Fig. 8. 

Fig. 11 is a block diagram which shows an ar- 
35 rangement of a main circuit in an image encoding ap- 
paratus including all methods in embodiments 1 
through 4. 

Fig. 12. is a graph which shows an aged change 
in a number of occupied bits in a buffer in the case 
40 where information corresponding to the image 2 and 
the image 3 is respectively encoded by the image en- 
coding apparatus shown in Fig. 11. 

Fig. 13 is an explanatory drawing which shows a 
format of dummy data generated from a dummy data 
45 generator in the image encoding apparatus shown in 
Fig. 1. 

Fig. 14 is an explanatory drawing which shows 
another format of the dummy data generated from the 
dummy data generator in the image encoding appa- 
so ratus shown in Fig. 1. 

Fig. 15 is an explanatory drawing which shows 
still another format of the dummy data generated 
from the dummy data generator in the image encod- 
ing apparatus shown in Fig. 1. 
55 Fig. 16 is a block diagram which shows an ar- 

rangement of a main circuit in a conventional image 
encoding apparatus. 

Fig. 17 is a graph which shows an aged change 
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in a number of occupied bits in a buffer in the case 
where a typical motion picture is encoded by the im- 
age encoding apparatus. 

Fig. 18 is a graph which shows an aged change 
in a number of occupied bits in a buffer in the case 
where information corresponding to the image 2 and 
the image 3 is respectively encoded by the conven- 
tional image encoding apparatus. 

DESCRIPTION OF THE EMBODIMENTS 

[EMBODIMENT 1] 

The following description will discuss one em- 
bodiment of the present invention mainly referring to 
Figs. 1 through 3 and Figs. 13 through 15. 

The image encoding apparatus of the present in- 
vention controls underflow of a buffer which is one of 
the problems in the prior arts, and an arrangement of 
its main part is shown in Fig. 1 . 

As shown in Fig. 1, the image encoding appara- 
tus includes a block dividing circuit 1, a motion com- 
pensation inter-frame predicting circuit 2, an orthogo- 
nal transforming circuit 3, a quantizing circuit 4 as 
quantizing means, a variable-length encoding circuit 
5 as variable-length encoding means, a buffer 6 as 
temporary storing means, an inverse quantizing cir- 
cuit 7, inverse orthogonal transforming circuit 8, a 
frame memory 9, a bit allocating circuit 10, a refer- 
ence quantization step size determining circuit 11, a 
quantization step size determining circuit 12, a sub- 
tracter 13, an adder 14 having two input terminals, a 
switch 15 for switching an input to a minus input ter- 
minal of the subtracter 13, a switch 16 for switching 
an input to one input terminal of the adder 14, a dum- 
my data generator 17 as dummy data generating 
means, a switch 18 as switching means for switching 
whether an output of the image encoding apparatus 
is performed from the buffer 6 or the dummy data 
generator 17, and a controller 1 9 as control means for 
controlling the switch 18 while supervising a number 
of occupied bits in the buffer 6. 

In the same manner as the prior arts, the image 
encoding apparatus divides a frame to be encoded 
into, for example, MBs of 16x16 pixels and intra-fra- 
me encoding or intra-frame predictive encoding is 
performed per MB. There exists a frame on which 
only intra-frame encoding is performed (I picture), a 
frame on which inter-frame predictive encoding is 
performed from previous frame (P picture), and a 
frame on which inter-frame predictive encoding is 
performed from previous and subsequent frames (B 
picture). The intra-frame encoding is performed at a 
predetermined frame interval. In the present embodi- 
ment, the intra-frame encoding is performed per 15 
frames (in other words, a number of frames in GOP 
N is 15), but the present invention is not limited to this, 
so a number of frames in the GOP N can be changed 



by setting. 

As shown in Fig. 3, a first image (frame) in the 
GOP is an I picture. Non intra prediction is performed 
on a P picture after a predetermined number of 

5 frames M from the I picture, and from the I picture and 
the P picture, inter-frame prediction is performed on 
a B picture between the I picture and the P picture. 
Moreover, inter-frame prediction is performed on a P 
picture after a predetermined number of frames M 

10 from P picture on which inter-frame prediction has 
been performed from the I picture, and from the both 
P pictures, inter-frame prediction is performed on B 
picture between the P pictures. In this manner, as to 
P picture, inter-frame prediction is performed from 

15 previous I picture or previous P picture, and as to B 
picture, inter-frame prediction is made from I picture 
or P picture before and after the B picture. As to an 
arrangement of the GOP in a number of frames N, a 
first frame in the GOP is an I picture, (1 + nM)th frame 

20 in the GOP is a P picture (however, 1 + nM ^ N, n is 
a natural number), frames other than the above 
frames in the GOP are B pictures. In the present em- 
bodiment, as shown in Fig. 3, an interval between an 
I picture and a P picture and an interval between P 

25 pictures M are 3, but the present invention is not lim- 
ited to this, so M may be 2 or not less than 4. 

As shown in Fig. 1, an original image is inputted 
to the block dividing circuit 1 (after in the case where 
the original image is analog, it is transformed to digi- 

30 tal), and a frame to be encoded is divided into, for ex- 
ample, MBs of 16x16 pixels so that the MBs are in- 
putted to a plus input terminal of the subtracter 1 3, the 
motion compensation inter-frame predicting circuit 2 
and the quantization step size determining circuit 12. 

35 The motion compensation inter-frame predicting 

circuit 2 compares a current frame with a frame pre- 
viously encoded and decoded, and while a moving 
amount of the MB (a moving vector) is detected, the 
MBs of P picture or B picture, in which inter-frame en- 

40 coding is advantageous, are predicted from another 
frame as mentioned above. However, the MB of I pic- 
ture and a MB in which intra-frame encoding is ad- 
vantageous are not predicted. 

In the case where an output of the block dividing 

45 circuit 1 is the MB of the I picture or the MB in which 
intra-frame encoding is advantageous, the switch 15 
is entered to an input terminal 15b side, and the MB 
of the I picture is inputted from the subtracter 13 di- 
rectly to the orthogonal transforming circuit 3. 

so Meanwhile, in the case where the output of the 

block dividing circuit 1 is the MB of P picture or B pic- 
ture, the switch 15 is entered to an input terminal 15a 
side, a MB from the block dividing circuit 1 and an pre- 
dicted MB from the motion compensation inter-frame 

55 predicting circuit 2 are respectively inputted to a plus 
and a minus input terminals of the subtracter 1 3, and 
a prediction difference (prediction error) is outputted 
from the subtracter 13 to the orthogonal transforming 
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circuit .3. 

Data from the subtracter 13 (MB or prediction dif- 
ference) are converted into data suitable for encoding 
by means of two-dimensional orthogonal transforma- 
tion (discrete Fourier transformation, Hadamard 
transformation, discrete cosine transformation, etc.) 
in the orthogonal transforming circuit 3, and the trans- 
formed data are outputted to the quantizing circuit 4. 

In the quantizing circuit 4, a result of the two-di- 
mensional orthogonal transformation (namely, a 
transform coefficient) is quantized by using a quanti- 
zation step size obtained in the quantization step size 
determining circuit 12, and the quantized result is out- 
putted to the variable-length encoding circuit 5 and 
the inverse quantizing circuit 7. 

In the variable-length encoding circuit 5, variable- 
length encoding is performed on the quantized trans- 
form coefficient, and the encoded transform coeffi- 
cient is outputted to the buffer 6 and the bit allocating 
circuit 10. 

The encoded data from the variable-length en- 
coding circuit 5 are temporarily stored in the buffer 6, 
and the encoded data are outputted at a predeter- 
mined constant data rate therefrom. 

The quantized transform coefficient is inversely 
quantized in the inverse quantizing circuit 7 and is 
outputted to the inverse orthogonal transforming cir- 
cuit 8. 

Reverse orthogonal transformation is carried out 
on the inversely quantized transform coefficient by 
the inverse orthogonal transforming circuit 8 so that 
a reconstructed value before the orthogonal transfor- 
mation (MB or prediction difference) is obtained. 

In the case where the reconstructed value is a 
prediction difference, the switch 1 6 is changed to an 
input terminal 16a side, and the prediction difference 
as a reconstructed value and the prediction MB out- 
putted from the motion compensation inter-frame 
predicting circuit 2 are added in the adder 14 so that 
an MB is reconstructed. Meanwhile, in the case 
where the reconstructed value is an MB on which In- 
tra-frame encoding is performed, the switch 16 is 
changed to an input terminal 16b side so that the re- 
constructed value is directly outputted to the frame 
memory 9. 

A reconstructed image obtained from the MB re- 
constructed in the above manner is stored in the 
frame memory 9. The reconstructed image is used as 
a reference image for inter-frame prediction in the 
motion compensation inter-frame predicting circuit 2. 

The bit allocating circuit 10, the reference quan- 
tization step size determining circuit 11 and the quan- 
tization step size determining circuit 12 control a gen- 
erating an amount of encoded data, and they have 
same functions and arrangements as those of a bit al- 
locating circuit 64, a reference quantization step size 
determining circuit 65, and a quantization step size 
determining circuit 66 in a conventional image encod- 



ing apparatus shown in Fig. 16. 

In other words, the bit allocating circuit 10 deter- 
mines a number of bits which are allocated to a frame 
to be encoded based upon encoded results up to the 

5 previous frame (a number of generated bits obtained 
from the variable-length encoding circuit 5, an aver- 
age value of a quantization step size obtained from 
the quantization step size determining circuit 12) and 
a given data rate. To be concrete, as shown in the pri- 

10 or art, respective global complexity measures X l , X P 
and X B of an I picture, a P picture and a B picture are 
determined according to the aforementioned formu- 
las (1) through (3), and allocations of target bits T, f T P 
and T B to images to be encoded are determined by 

15 calculating the aforementioned formulas (4) through 
(6) for I picture, P picture and B picture. Here, at the 
beginning of encoding, fixed initial values are set for 
the respective global complexity measures X,, X P and 
X B - 

20 The reference quantization step size determining 

circuit 11 determines a reference quantization step 
size of a orthogonal transformation coefficient per 
MB based upon a number of generated bits up to the 
previous MB obtained from the variable-length en- 

25 coding circuit 5 and a number of bits allocated by the 
bit allocating circuit 10. For example, in the case of I 
picture, a virtual buffer fullness d/ in the j-th MB of the 
I picture is decided according to the aforementioned 
formula (7), and a reference quantization step size 

30 is determined per MB according to the aforemen- 
tioned formula (8). Furthermore, also in the case of P 
picture and B picture, reference quantization widths 
qj p and qj B of j-th MB are determined in the same man- 
ner as in the case of I picture. Here, at the beginning 

35 of encoding, initial buffer fullness d Q ', d Q p and d Q B of 
respective pictures (I picture, P picture and B picture) 
are initialized. 

The quantization step size determining circuit 12 
calculates an activity for each MB which shows f ine- 

40 ness of a picture in a MB obtained from the block div- 
iding circuit 1 , a reference quantization step size is 
changed by the activity per MB so as to determine a 
quantization step size which is used in the quantizing 
circuit 4 per MB. For example, in the case of I picture, 

45 a quantization step size Q/ of j-th MB of the I picture 
is determined according to the aforementioned for- 
mula (9). Moreover, also in the case of P picture and 
B picture, quantization widths Qf and Qj B of j-th MB 
are determined in the same manner as in the case of 

so the I picture. Here, at the beginning of encoding, an 
average value (avg) of the activities of the previously 
encoded frame is initialized. 

Here, in the present embodiment, the bit allocat- 
ing circuit 10, the reference quantization step size de- 

55 termining circuit 11 and the quantization step size de- 
termining circuit 12 constitute encoded data amount 
control means. 

The dummy data generator 17, the switch 18 and 
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the controller 19 realizes prevention of underflow in 
the buffer 6 which is one of the objects of the present 
invention. 

The controller 19 monitors a number of occupied 
bits in the buffer 6 (a number of bits of encoded data 5 
which remain in the buffer 6 in the case where the en- 
coded data are outputted at a- constant data rate after 
each image frame is immediately encoded and the 
encoded data are immediately stored in the buffer 6 
in the image encoding apparatus) and switches the 10 
switch 18 to an input terminal 18a side so that the out- 
put of the buffer 6 becomes an output of the image en- 
coding apparatus (encoded data) in the case where 
a number of occupied bits exceeds a fixed value Th1 , 
and switches the switch 18 to an input terminal 18b 15 
side so that an output of the dummy data generator 
17 becomes the output of the image encoding appa- 
ratus in the case where a number of occupied bits is 
below the fixed value Th1. 

The dummy data generator 1 7 generates dummy 20 
data which can be discriminated on a decoder side, 
not shown, and be eliminated. For example, in the 
case where "dummy data start code" and "dummy 
data end code" are provided in a variable-length code 
list and when the switch 18 is switched to the dummy 25 
data generator 1 7 side, the dummy data generator 1 7 
generates an arbitrary bit string (however, the bit 
string does not form "dummy data end code") succes- 
sively after generating "dummy data start code", and 
generates "dummy data end code" just before the 30 
switch 18 is switched to the buffer 6 side. A format of 
the dummy data is shown in Fig. 13. 

Or, a dummy code composed of a plurality of bits 
is set, and while the switch 18 is on the dummy data 
generator 17 side, the dummy data generator 17 may 35 
generate successively the above kind of the dummy 
codes. A format of the dummy data is shown in Fig. 
14. 

Or, "start code" is always inserted into the front 
of each frame. In the case of a encoding system which 40 
is capable of inserting bits, an arbitrary number of 
"0"s just after the "start code" (namely, a system 
where on the image encoding apparatus side, the 
"start code" is always inserted into the front of encod- 
ed data of each frame, and on the decoder side, bits 45 
"0" just after the "start code" (bit string) are eliminat- 
ed), the controller 19 and the dummy data generator 
17 may have the following functions. Namely, the 
controller 19 checks whether or not the "start code" 
is inputted from the variable-length encoding circuit 5 50 
to the buffer 6, and monitors a number of occupied 
bits in the buffer 6 when the "start code" is inputted 
to the buffer 6. When a number of occupied bits in the 
buffer 6 is below a determined value Thl and the "start 
code" is outputted from the buffer 6, the switch 1 8 is 55 
switched to the dummy data generator 17 side, and 
when not, switched to the buffer 6 side. The dummy 
data generator 17 successively generates bits "0" 



(successively outputs the same bits). In this manner, 
a plurality of bits "0" continuously exist just after the 
"star code" in the encoded data, but they can be elim- 
inated on the decoder side. A format of the dummy 
data is shown in Fig. 15. 

The following will discuss an operation of the im- 
age encoding apparatus having the above arrange- 
ment. 

First, in the block dividing circuit 1, a frame to be 
encoded is divided into MBs, and intra-frame encod- 
ing or inter-frame encoding is performed per MB. In 
the case of the intra-frame encoding, two-dimension- 
al orthogonal transformation is carried out on the MBs 
from the block dividing circuit 1 by the orthogonal 
transforming circuit 3, whereas in the case of the in- 
ter-frame encoding, two-dimensional transformation 
is carried out on a difference (prediction difference) 
between the MBs from the block dividing circuit 1 and 
prediction MBs from the motion compensation inter- 
frame predicting circuit 2 by the orthogonal trans- 
forming circuit 3. 

Next, in the quantizing circuit 4, a result of the 
two-dimensional orthogonal transformation by the 
orthogonal transform circuit 3 (transformation coeffi- 
cient) is quantizing by a quantization step size ob- 
tained in the quantization step size determining circuit 
12. After that, in the variable-length encoding circuit 
5, variable-length encoding is performed on the quan- 
tized transform coefficient and the encoded data are 
temporarily stored in the buffer 6. 

The controller 19 monitors a number of occupied 
bits in the buffer 6, and when the number of occupied 
bits in the buffer 6 exceeds a threshold Thl, a com- 
mon output terminal of the switch 18 is connected to 
the input terminal 18a. In this case, the encoded data 
are outputted from the buffer 6 at a predetermined 
constant rate. 

Fig. 2 shows a number of occupied bits in the buf- 
fer 6 in the case where an image 3 is encoded by the 
image encoding apparatus of the present embodi- 
ment (image that is shifted to a normal motion picture 
after frequent scene changes for continuous 30 
frames). In this example, a threshold Th1 which 
changes the switch 18 is -10 5 bits. 

In this case, the number of occupied bits in the 
buffer 6 is not more than the threshold Th1 at about 
time 27, and the switch 18 is switched to the dummy 
data generator 17 side by the controller 19 which 
monitors the number of occupied bits in the buffer 6. 
Therefore, from the image encoding apparatus, in- 
stead of the encoded data read out from the buffer 6, 
dummy data from the dummy data generator 17 are 
outputted. While the switch 18 is on the dummy data 
generator 17 side, transmission of the encoded data 
in the buffer 6 is temporarily suspended, so the num- 
ber of occupied bits in the buffer 6 does not become 
smaller than the threshold Th1. 

While the switch 18 is on the dummy data gener- 
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ator 17 side, the encoded data from the variable- 
length encoding circuit 5 are stored in the buffer 6. If 
a number of occupied bits in the buffer 6 exceeds the 
threshold Th1 , the switch 18 is switched to the buffer 
6 side by the controller 19, and the encoded data are 
again outputted from the buffer at a constant data 
rate. 

Thereafter, as mentioned above, if a number of 
occupied bits in the buffer 6 is not more than the 
threshold Th1 , an output of the image encoding appa- 
ratus becomes dummy data of the dummy data gen- 
erator 17, and if a number of occupied bits in the buf- 
fer 6 exceeds the threshold Th1 , the encoded data are 
outputted from the buffer 6 at a constant data rate. 
For this reason, a lower limit of a number of occupied 
bits in the buffer 6 can be limited substantially to the 
threshold Th1, thereby.making it possible to prevent 
underflow of the buffer 6. 

Here, allowing a number of occupied bits to be a 
minus value corresponds to specifying an origin of 
the occupied bits in the following manner. Namely, in 
the initial state, dummy data which can be eliminated 
on the decoder side are stored in the buffer 6, and a 
number of occupied bits at this time is defined as 0. 
Therefore, the threshold Th1 is allowed to be set to a 
smaller value than a number of occupied bits in the 
buffer 6 in the initial state. Generally, also in the case 
of a normal image, the rate control (control over an 
amount of generation of encoded data) is not accu- 
rate, namely, slight fluctuation occur. However, when 
the threshold Th1 is set to 0, even in the case where 
the rate control is slightly inaccurate, the switch 18 is 
switched to the dummy data generator 1 7 side and the 
dummy data are added to the encoded data. Reason 
for setting the threshold Th1 to a minus is to avoid the 
addition of such dummy data. 

Here, only if a lot of dummy data are stored in the 
buffer 6 and an origin is set so that a number of oc- 
cupied bits become a great minus value in the early 
state, the problem of underflow mentioned in prob- 
lems to be solved by the invention cannot be essen- 
tially dissolved. The reason is that, for example, in a 
sequence where deep black images continue for a 
long time, the encoded data are hardly generated, so 
an amount of dummy data which is stored in the buffer 
6 at the beginning should be greatly increased. In this 
case, since capacity of the buffer 6 should be en- 
larged greatly, there arises a problem that scale of 
hardware increases. Furthermore, in a sequence of 
an image 2 (an image which is shifted to a normal mo- 
tion picture after stop motion continues for 30 frames) 
or the image 3, the underflow can be prevented, but 
a problem that qua! ity of the image is considerably de- 
teriorated at that time cannot be dissolved. 

Therefore, an operation for obtaining minus val- 
ues for a number of occupied bits in the buffer 6 ab- 
sorbs slight fluctuation of the rate control in a normal 
sequence so that unnecessary addition of the dummy 



data by the dummy data generator 17 in the present 
embodiment is prevented. However, needless to say, 
the threshold Th1 may be 0 as well as a positive val- 
ue. In the case particularly where the fluctuation of 
5 the rate control in the normal sequence is small, the 
threshold Th1 may be 0 as well as a positive value. 

Here, in the case where a normal motion picture 
(image 1) in which an stop motion and frequent scene 
change do not continue for a plurality of frames is en- 
10 coded by the image encoding apparatus of the pres- 
ent embodiment, a number of occupied bits in the buf- 
fer 6 seldom becomes not more than the threshold 
Th 1 , so similarly to the conventional i mage encoding 
apparatus, a change in a number of occupied bits in 
15 the buffer 6 with time is shown in Fig. 17. 

As mentioned above, the image encoding appa- 
ratus of the present embodiment is arranged so as to 
have: 

the quantizing circuit 4 as quantizing means for 
20 quantizing orthogonal transform coefficient as infor- 
mation corresponding to image Information; 

the variable-length encoding circuit 5 as the 
variable-length encoding means for performing vari- 
able-length encoding on the quantized result; 
25 the buffer 6 as temporary storage means for 

temporarily storing encoded data from the variable- 
length encoding circuit 5 so as to transmit the encod- 
ed data, which have been stored, outside at a fixed 
data rate; 

30 the encoded data amount control means (bit 

allocating circuit 10, reference quantization step size 
determining circuit 11 and quantization step size de- 
termining circuit 12) for calculating a quantization 
step size of information to be encoded based upon 

35 the encoded result of the variable-length encoding 
circuit 5 adjusting the quantization step size for quan- 
tization in the quantizing circuit 4 according to the cal- 
culated result so as to control an amount of the encod- 
ed data; 

40 the dummy data generator 17 as dummy data 

generating means for generating dummy data which 
can be eliminated on the decoding circuit which de- 
codes the encoded data; 

the switch 18 as switching means for switching 

45 an output of the image encoding apparatus to either 
the encoded data from the buffer 6 or the dummy data 
from the dummy data generator 17; and 

the controller 1 9 as control means for monitor- 
ing a number of occupied bits in the buffer 6 so as to 

50 control the switch operation of the switch 18 such that 
the output of the image encoding apparatus becomes 
the encoded data from the buffer 6 when a number 
of occupied bits exceeds the predetermined thresh- 
old Th1 and that the output of the image encoded ap- 

55 paratus becomes the dummy data from the dummy 
data generator 17 when a number of occupied bits is 
not more than the threshold Th1 . 

With the above arrangement, when a number of 

10 



BNSDOCID: <EP 066061 2A2J_> 



19 



EP 0 660 612 A2 



20 



occupied bits in the buffer 6 becomes smaller than 
the threshold Thl, the dummy data from the dummy 
data generator 17 are outputted from the image en- 
coded apparatus instead of the encoded data read 
out from the buffer 6. Since while the dummy data are 
being outputted, transmission of the encoded data in 
the buffer 6 is temporarily suspended, a lower limit of 
a number of occupied bits in the buffer 6 can be lim- 
ited to the threshold Th1, thereby making it possible 
to prevent the underflow of the buffer 6. 

[EMBODIMENT 2] 

Next, the following will discuss another embodi- 
ment of the present invention in detail mainly referring 
to Figs. 4 and 5. 

Here, for convenience of explanation, those 
members of the present embodiment that have the 
same arrangement and function, and that are men- 
tioned in the above example are indicated by the 
same reference numerals and the description thereof 
is omitted. 

Similarly to the aforementioned embodiment 1 , 
the present embodiment controls underflow of a buf- 
fer. According to the embodiment 1 f underflow of a 
buffer in an image encoding apparatus can be pre- 
vented, but since the encoded data stored in the buf- 
fer are increased by inserting dummy data despite of 
actually little generation of the encoded data, various 
parameters which are used for the rate control (con- 
trolling an amount of generation of the encoded data) 
lose their balance. For example, since global com- 
plexity measures X,, X P and Xb in I picture, P picture 
and B picture are calculated from a number of gener- 
ated bits and an average quantization step size of I 
picture, P picture and B picture, if the dummy data are 
inserted into the encoded data, the global complexity 
measures become meaningless values. For this rea- 
son, in an example shown in Fig. 2, just after the fre- 
quent scene change is shifted to a normal motion pic- 
ture (just after time 30), rate control is unstable, so it 
takes too time to return to the normal state which is 
shown in Fig. 17. Actually, turbulence in quality of an 
image can be seen. The present embodiment dis- 
solves these problems. 

Fig. 4 shows an arrangement of the main portion 
of the image encoding apparatus according to the 
present embodiment. A block dividing circuit 1, a mo- 
tion compensation inter-frame predicting circuit 2, an 
orthogonal transforming circuit 3, a quantizing circuit 
4, a variable-length encoding circuit 5, a buffer 6, an 
inverse quantizing circuit 7, an inverse orthogonal 
transforming circuit 8, a frame memory 9, a subtract- 
er 13, an adder 14, a switch 15 and a switch 16 in the 
image encoding apparatus of the present embodi- 
ment that have the same arrangement and functions, 
and that are mentioned in the above embodiment 1 
are indicated by the same reference numerals and the 



description thereof is omitted. 

A bit allocating circuit 210, a reference quantiza- 
tion step size determining circuit 211, a quantization 
step size determining circuit 212, a dummy data gen- 

5 erator 217 as dummy data generating means, a 
switch 218 as switching means and a controller 219 
as control means in the image encoding apparatus of 
the present embodiment adopt the method of the 
present invention so as to prevent underflow in the 

w buffer 6 of the image encoding apparatus, and have 
a function for keeping good quality of an image. 

The bit allocating circuit 21 0, the reference quan- 
tization step size determining circuit 211, the quanti- 
zation step size determining circuit 212, the dummy 

15 data generator 217, the switch 218 and the controller 
219 basically have same functions as those of the the 
bit allocating circuit 10, the reference quantization 
step size determining circuit 11, the quantization step 
size determining circuit 12, the dummy data generator 

20 17 and the switch 18 and the controller 19 of the 
aforementioned embodiment 1 which are shown in 
Fig. 1, but in the present embodiment, the following 
operations are further added. 

In other words, the controller 219 switches the 

25 switch 218 to the dummy data generator 217 side and 
simultaneously transmits a switch signal a which 
shows this operation to the bit allocating circuit 210, 
the reference quantization step size determining cir- 
cuit 211 and the quantization step size determining 

30 circuit 212. 

In the bit allocating circuit 210, the reference 
quantization step size determining circuit 21 1 and the 
quantization step size determining circuit 212, when 
the switch signal a is inputted from the controller 21 9, 

35 a parameter which is used for the rate control is ini- 
tialized. In other words, when the switch signal a is in- 
putted to the bit allocating circuit 210, the global com- 
plexity measures X,, X P and Xb of each picture (I pic- 
ture, P picture and B picture) are initialized (set to an 

40 initial value). Moreover, when the switch signal a is in- 
putted to the reference quantization step size deter- 
mining circuit 211, initial buffer fullness do 1 , d 0 p , d 0 B of 
each picture (I picture, P picture and B picture) are ini- 
tialized (set to an initial value). Furthermore, when the 

45 switch signal a is inputted to the quantization step 
size determining circuit 212, an average value avg of 
the activities in the previous frame is initialized (set 
to an initial value). 

Here, in the present embodiment, the bit allocat- 

50 ing circuit 210, the reference quantization step size 
determining circuit 211 and the quantization step size 
determining circuit 212 constitute the encoded data 
amount control means. 

With the above arrangement, the following will 

55 discuss an operation of the image encoding appara- 
tus of the present embodiment. 

As mentioned in the aforementioned embodi- 
ment 1, the controller 219 monitors a number of oc- 

11 
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cupied bits in the buffer 6 and judges whether or not 
the number of occupied bits in the buffer 6 exceeds 
the threshold Th1. When the number of occupied bits 
in the buffer 6 exceeds the threshold Th1 , the control- 
ler 219 switches the switch 218 to the buffer 6 side 
and encoded data are outputted from the buffer 6 at 
a predetermined constant data rate. 

Fig. 5 shows a number of occupied bits in the buf- 
fer 6 in the case where an image 3 (an image which 
is shifted to a normal motion picture after frequent 
scene change continues for 30 frames) is encoded by 
the image encoding apparatus of the present embodi- 
ment. Also in this case, the threshold Th1 which 
switches the switch 218 is -10 5 bits. 

In this case, the number of occupied bits in the 
buffer 6 becomes not more than the threshold Thl at 
about time 27. Then, the switch 21 8 is switched to the 
dummy data generator 21 7 side by the controller 21 9 
which monitors the number of occupied bits in the 
buffer 6 and at the same time, the switch signal a is 
outputted from the controller 219 to the bit allocating 
circuit 21 0, the reference quantization step size deter- 
mining circuit 211 and the quantization step size de- 
termining circuit 212. 

Therefore, dummy data from the dummy data 
generator 217 are outputted from the image encoding 
apparatus instead of the encoded data read out from 
the buffer 6, and while the dummy data is being out- 
putted, transmission of the encoded data in the buffer 
6 is suspended, so the number of occupied bits in the 
buffer 6 does not become less than the threshold Th1 . 
Moreover, when an output of the dummy data is start- 
ed, a parameter which is used for the rate control is 
initialized iri the bit allocating circuit 210, the refer- 
ence quantization step size determining circuit 211 
and the quantization step size determining circuit 
212. For this reason, even when the dummy data are 
inserted into the encoded data, the parameter for the 
rate control, which is used for calculating a quantiza- 
tion step size, can keep its balance. 

While the switch 218 is on the dummy data gen- 
erator 217 side, the encoded data from the variable- 
length encoding circuit 5 are stored in the buffer 6. If 
the number of occupied bits in the buffer 6 exceeds 
the threshold Thl, the switch 218 is switched to the 
buffer 6 side by the controller 219 and the encoded 
data are again outputted from the buffer 6 at a con- 
stant data rate. 

Subsequently, in the same manner as in the 
above, if the number of occupied bits in the buffer 6 
becomes not more than the threshold Th1, transmis- 
sion of the encoded data stored in the buffer 6 is sus- 
pended so that the dummy data are outputted instead 
of the encoded data and a parameter which is used 
for the rate control is initialized. Thereafter, if the 
number of occupied bits in the buffer 6 exceeds the 
threshold Thl, the encoded data are outputted from 
the buffer 6 at a constant rate. For this reason, a lower 



limit of the number of occupied bits in the buffer 6 can 
be substantially limited to the threshold Th1 . As a re- 
sult, the underflow of the buffer 6 can be prevented 
and also the parameter for the rate control during the 

5 insertion of the dummy data can keep its balance. 

It is apparent from Fig. 5 that the underflow of the 
buffer 6 can be prevented, and the normal rate control 
is worked just after the image is shifted to the normal 
motion picture. In other words, in the aforementioned 

10 embodiment 1 , as shown in Fig. 2, the number of oc- 
cupied bits in the buffer 6 becomes repeatedly not 
more than the threshold Th1 even after time 30, and 
the dummy data are inserted by the present invention. 
However, in the present embodiment, as shown in 

15 Fig. 5, a configuration of the graph just after time 30 
is same as that of the normal motion picture (Fig. 17), 
namely, it is returned to a normal state. 

Similarly to the image encoding apparatus of the 
embodiment 1 , the image encoding apparatus of the 

20 present embodiment is arranged so as to have: 

the dummy data generator 21 7 as dummy data 
generating means for generating dummy data which 
can be eliminated on the decoding circuit which de- 
codes encoded data; 

25 the switch 218 as switching means for switch- 

ing an output of the image encoding apparatus to eith- 
er the encoded data from the buffer 6 or the dummy 
data from the dummy data generator 217; and 

the controller 219 as control means for moni- 

30 toring a number of occupied bits in the buffer 6 and 
controlling the switching operation of the switch 218 
so that the output of the image encoding apparatus 
becomes the encoded data from the buffer 6 when a 
number of occupied bits exceeds the predetermined 

35 thresholdThl and that the output of the imageencod- 
ed apparatus becomes the dummy data from the 
dummy data generator 217 when a number of occu- 
pied bits is not more than the threshold Th1 . Further- 
more, the image encoding apparatus of the present 

40 embodiment is arranged so that when the number of 
occupied bits in the buffer 6 is not more than the 
threshold Th1, the switch signal a is outputted to the 
bit allocating circuit 210, the reference quantization 
step size determining circuit 211 and the quantization 

45 step size determining circuit 21 2 as the encoded data 
amount control means by the controller 219, and 
when the switch signal a is inputted, the bit allocating 
circuit 21 0, the reference quantization step size deter- 
mining circuit 211 and the quantization step size de- 

50 termining circuit 212 initialize various parameters for 
the rate control (various parameters which are used 
for calculating a quantization step size). 

With the above arrangement, the underflow of 
the buffer 6 can be prevented in the same way as in 

55 the embodiment 1, and even when the dummy data 
are outputted instead of the actual encoded data, va- 
rious parameters which are used for the rate control 
keep their balance and the constant rate control can 

12 
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be realized, thereby making it possible to code image 
information with good quality. 

[EMBODIMENT 3] 

Next, the following will discuss a third embodi- 
ment of the present invention mainly referring to Figs. 
6 and 7. 

Here, for convenience of explanation, those 
members of the present embodiment that have the 
same arrangement and function, and that are men- 
tioned in the above embodiments are indicated by the 
same reference numerals and the description thereof 
is omitted. 

The image encoding apparatus of the present 
embodiment controls overflow of a buffer which is 
one of the problems in the prior arts, and an arrange- 
ment of its main circuit is shown in Fig. 6. 

The block dividing circuit 1, the motion compen- 
sation inter-frame predicting circuit 2, the orthogonal 
transforming circuit 3, the quantizing circuit 4, the va- 
riable-length encoding circuit 5, the buffer 6, the in- 
verse quantizing circuit 7, the inverse orthogonal 
transforming circuit 8, the frame memory 9, the bit al- 
locating circuit 10, the reference quantization step 
size determining circuit 11, the quantization step size 
determining circuit 12, the subtracter 13, the adder 
14, the switch 15 and the switch 16 in the image en- 
coding apparatus of the present embodiment that 
have the same arrangement and functions, and that 
are mentioned in the above embodiment 1 are indicat- 
ed by the same reference numerals and the descrip- 
tion thereof is omitted. 

The image encoding apparatus of the present 
embodiment includes a bit allocation changing circuit 
301 as bit allocation changing means in order to pre- 
vent overflow of a buffer. When bit allocation T ( (in the 
case of I picture), T P (in the case of P picture) or T B 
(in the case of B picture) is inputted from a bit allocat- 
ing circuit 10, the bit allocation changing circuit 301 in- 
spects a number of occupied bits in a buffer 6 and 
changes the bit allocation so that a sum of the number 
of occupied bits in the buffer 6 and the bit allocation 
T,, T P or T B for a frame to be encoded does not exceed 
a predetermined fixed value. 

In other words, when the bit allocation T t (in the 
case of I picture), T P (in the case of P picture) or T B 
(in the case of B picture) for a frame is inputted from 
the bit allocating circuit 10, a check is made as to 
whether or not conditions of the following formulas 
(11) through (13) are satisfied in the bit allocation 
changing circuit 301. 

BO+T, > Thl ... (11) (in the case of I picture) 
BO+Tp > ThP ... (12) (in the case of P picture) 
BO+T B > ThB ... (13) (in the case of B picture) 

Here, in the formulas (11) through (13), BO is a 
number of occupied bits in the buffer 6, and Thl, ThP 



or ThB is a threshold which is determined for each pic- 
ture. 

When the conditions of the formulas (11) through 
(13) are not satisfied, the bit allocation changing cir- 

5 cuit 301 outputs a value of the bit allocation, whereas 
when the conditions of the formulas (1 1 ) through (1 3) 
are satisfied, the bit allocation changing circuit 301 
changes the values of the bit allocation T, (in the case 
of I picture), T P (in the case of P picture) or T B (in the 

10 case of B picture) are changed according to the fol- 
lowing formulas (14) through (16). 

T, = Thl-BO ... (14) (in the case of I picture) 
T P = ThP-BO ... (15) (in the case of P picture) 
T B = ThB-BO ... (16) (in the case of B picture) 

15 With the above arrangement, the following will 

discuss an operation of the image encoding appara- 
tus of the present embodiment. 

Every time when encoding of one frame is com- 
pleted, in the bit allocating circuit 1 0, a number of bits 

20 which are allocated to a frame to be encoded, namely, 
the bit allocation T| (in the case of I picture), Tp (in the 
case of P picture) or TB (in the case of B picture) is 
calculated according to encoded results up to the pre- 
vious frame (a number of generated bits obtained 

25 from the variable-length encoding circuit 5 and an 
average value of a quantization step size obtained 
from the quantization step size determining circuit 
12), and the calculated value is outputted to the bit al- 
location changing circuit 301 . 

30 When the bit allocation is inputted, a check is 

made whether or not the conditions of the formulas 
(11) through (13) are satisfied in the bit allocation 
changing circuit 301. If the conditions are not satis- 
fied, the value of the bit allocation is directly outputted 

35 to the reference quantization step size determining 
circuit 11 , and if the conditions are satisfied, the value 
of the bit allocation is changed according to the for- 
mulas (14) through (16) and the changed bit alloca- 
tion is outputted to the reference quantization step 

40 size determining circuit 11 . 

Fig. 7 shows a number of occupied bits in the buf- 
fer 6 in the case where an image 2 (an image which 
is sh if ted to a normal motion picture after stop motion 
continues for 30 frames) is encoded by the image en- 

45 coding apparatus of the present embodiment In this 
example, all thresholds Thl, ThP and ThB are set to 
8x1 0 5 . 

In this case, since new GOP starts in a stop mo- 
tion at time 15 and an I picture appears, the bit allo- 

so cation Tl which is calculated in the bit allocating circuit 
1 0 becomes great unusually and BO + T, exceeds the 
threshold Thl. As a result, the condition of the formula 
(11) is satisfied. For this reason, the bit allocation 
changing circuit 301 changes the bit allocation ac- 

55 cording to the formula (14) so that BO + T| does not 
exceed the threshold Thl. Therefore, even if the I pic- 
ture which appears at time 1 5 is encoded and stored 
to the buffer 6, the number of occupied bits in the buf- 
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fer 6 does not considerably exceed the threshold Thl, 
so the overflow Is prevented. 

In addition, since generation of a lot of encoded 
data in an I picture which appears at time 15 are re- 
strained, a number of remained bits in GOP can be 
secured, and an usual number of bits is allocated to 
the frames thereafter in the GOP. Therefore, it is also 
possible to restrain deterioration of quality of an im- 
age. 

Effects of prevention of the overflow are quite ob- 
vious by comparing the graphs (solid line) for the im- 
age 2 in Figs. 7 and 18. 

As mentioned above, the image encoding appa- 
ratus of the present invention is arranged so as to 
have the bit allocation changing circuit 301 as the bit 
allocation changing means for monitoring a number of 
occupied bits in the buffer 6 so as to change the cal- 
culated bit allocation when the sum of bit allocation to 
a frame which has been calculated before encoding 
and a number of occupied bits in the buffer 6 at the 
time of the calculation of bit allocation exceeds the 
predetermined threshold Thl, ThP or ThB, such that 
the sum does not exceed the threshold. 

For this reason, even if information of the frame 
is encoded and stored to the buffer 6, a number of oc- 
cupied bits in the buffer 6 at that time does not con- 
siderably exceed the thresholds, so if the thresholds 
are set to smaller values than a storage capacity of 
the buffer 6, the overflow can be prevented. 

[EMBODIMENT 4] 

The following will discuss a fourth embodiment of 
the present invention mainly referring to Figs. 8 
through 10. 

Here, for convenience of explanation, those 
members of the present embodiment that have the 
same arrangement and function, and that are men- 
tioned in the above embodiments are indicated by the 
same reference numerals and the description thereof 
is omitted. 

Similarly to the embodiment 3, the image encod- 
ing apparatus of the present embodiment also con- 
trols overflow of a buffer, but further improves the ef- 
fects of the prevention of overflow in embodiment 3. 

An arrangement of a main circuit in the image en- 
coding apparatus of the present embodiment is 
shown in Fig. 8. A block dividing circuit 1, a motion 
compensation inter-frame predicting circuit 2, an or- 
thogonal transforming circuit 3, a quantizing circuit 4, 
a variable-length encoding circuit 5, a buffer 6, an in- 
verse quantizing circuit 7, an inverse orthogonal 
transforming circuit 8, a frame memory 9, a bit allo- 
cating circuit 10, a reference quantization step size 
determining circuit 11, a quantization step size deter- 
mining circuit 12, a subtracter 13, an adder 14, a 
switch 15, a switch 16 and a bit allocation changing 
circuit 301 in the image encoding apparatus of the 



present embodiment that have the same arrange- 
ment and functions, and that are mentioned in the 
above embodiment 3 are indicated by the same ref- 
erence numerals and the description thereof is omit- 
s ted. 

The image encoding apparatus of the present 
embodiment have a quantization step size changing 
circuit 401 as quantization step size changing means 
in order to effectively prevent overflow of the buffer 

10 6. When data of a quantization step size are inputted 
from the quantization step size determining circuit 12, 
the quantization step size changing circuit 401 in- 
spects a number of occupied bits in the buffer 6, and 
if the number of occupied bits in the buffer 6 exceed 

15 predetermined threshold, the quantization step size 
changing circuit 401 changes the quantization step 
size which has been determined in the quantization 
step size determining circuit 12 to a greater value. 
In other words, the quantization step size chang- 

20 ing circuit 401 changes the quantization step size by 
multiplying a value a shown in the following formula 
(17) or (18) by quantization widths Qj', Qj p and Qj B in 
j-th MB of each picture which have been determined 
in the quantization step size determining circuit 12. 

25 a = 1 + k (BO - C) (when BO > C) (17) 

a = 1 (when BO ^ C) (18) 
In the formula (17), k and C are constants (k > 0, 
C > 0). Arelationship between a and a number of oc- 
cupied bits BO is shown in Fig. 9. As shown in the ex- 

30 ample, a may linearly increase when BO is bigger 
than C but it is not limited to this. For example, a may 
be represented by a curve such that a gradually be- 
comes bigger than 1 with increase in BO when BO is 
smaller than C, and rapidly becomes bigger than 1 

35 when BO exceeds C. 

In the above arrangement, the following will dis- 
cuss an operation of the image encoding apparatus 
of the present embodiment 

In the same manner as in embodiment 3, the bit 

40 allocation T,, T P or T B calculated in the bit allocating 
circuit 10 is inputted, the bit allocating changing cir- 
cuit 301 changes a value of the bit allocation so that 
the sum of a number of occupied bits in the buffer 6 
and the bit allocation T,, T P or T B does not exceed a 

45 predetermined constant value. 

Thereafter, in the reference quantization step 
size determining circuit 11 and the quantization step 
size determining circuit 12, quantization widths are 
calculated per MB based upon the value of the bit al- 
so location from the bit allocation changing circuit 301 
and activity per MB, and the calculated results are 
outputted to the quantization step size changing cir- 
cuit 401. 

In the quantization step size changing circuit 401 , 
55 a number of occupied bits in the buffer 6 BO is 
checked every time when the data of a quantization 
step size are inputted from the quantization step size 
determining circuit 12, and when the number of occu- 
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pied bits in the buffer 6 BO exceeds the threshold C, 
the quantization widths are increased by multiplying 
a obtained by the calculation of the formula (17) by 
the quantization widths. 

Fig. 10 shows a number of occupied bits in the 
buffer 6 in the case where an image 2 (an image 
which is shifted to a normal motion picture after the 
stop motion continues for 30 frames) is encoded by 
the image encoding apparatus of the present embodi- 
ment. In this example, k = 3.1 x lO^ 4 , C = 8 x 10 5 , and 
Thl, ThP and ThB are all 8 x 10 5 . From Fig. 10, it is 
obvious that overflow of the buffer 6 is prevented 
more efficiently than in embodiment 3. 

As mentioned above, the image encoding appa- 
ratus of the present embodiment having the arrange- 
ment of embodiment 3 is arranged so as to further in- 
clude the quantization step size changing circuit 401 
as the quantization step size changing means for 
monitoring a number of occupied bits in the buffer 6 
BO so as to change the quantization widths calculat- 
ed in the quantization step size determining circuit 12 
to a greater value by multiplying a by the quantization 
widths when the number of occupied bits BO exceeds 
the predetermined threshold C. 

Therefore, not only the bit allocation is changed 
as mentioned in embodiment 3, when the number of 
occupied bits in the buffer 6 BO exceeds the thresh- 
old C, but also the quantization step size is changed 
to a greater value, thereby making it possible to pre- 
vent the overflow in the buffer 6 more efficiently than 
in embodiment 3. 

Here, in the present embodiment, the quantiza- 
tion step size is increased by multiplying a by the 
quantization step size, but the present embodiment is 
not limited to this, so the quantization step size may 
be increased by adding a certain value or changing a 
function f (Q) (Q is a quantization step size) where the 
relationship: f(Q) > Q holds. Needless to say, the 
processes for increasing the quantization step size 
are performed in the case of BO > C. 

In embodiments 1 through 4, a encoding method 
using arts, such as orthogonal transformation, mo- 
tion compensation inter-frame prediction, is adopted, 
but the present invention can be applied to a encoding 
method using arts other than the above. 

In addition, all or parts of the methods described 
in embodiments 1 through 4 are used simultaneously 
so that underflow or overflow of a buffer can be pre- 
vented. For example, Fig. 11 is a block diagram which 
shows an example of a encoding method including all 
the methods mentioned in embodiments 1 through 4. 
The image encoding apparatus in Fig. 11 that have 
the same arrangement and function, and that are 
mentioned in the above embodiments 1 through 4 are 
indicated by the same reference numerals and the de- 
scription thereof is omitted. 

A solid line and a broken line in Fig. 1 2 respective- 
ly show a number of occupied bits in the buffer 6 in 



the case where the image 2 and the image 3 are en- 
coded by the image encoding apparatus in Fig. 11. 
Here, the threshold Thl which makes the controller 
219 change the switch 21 8 is -1 0 5 bits, all the thresh- 

5 olds Thl, ThP and ThB of each picture which becomes 
a reference when the bit allocation per frame is 
changed are 8 x 1 0 5 , and the constant k in the formula 
(17) is 3.1 x 1f>^ and C is 8 x 10 5 . From Fig. 12, it is 
obvious that the underflow and overflow of the buffer 

10 6 is prevented efficiently. 

The invention being thus described, it will be ob- 
vious that the same may be varied in many ways. 
Such variations are not to be regarded as a departure 
from the spirit and scope of the invention, and all such 

15 modifications as would be obvious to one skilled in 
the art are intended to be included within the scope 
of the following claims. 



20 Claims 

1. An image encoding apparatus, comprising: 

quantizing means for quantizing informa- 
tion which corresponds to an image; 

25 variable-length encoding means for per- 

forming variable-length encoding on the quan- 
tized result of said quantizing means; 

temporary storage means for temporarily 
storing encoded data from said variable-length 

30 encoding means so as to transmit the stored en- 

coded data to an outside at a fixed data rate; 

encoded data amount control means for 
calculating a quantization step size of quantizing 
information to be encoded based upon the en- 

35 coded result of said variable-length encoding 

means so as to control an amount of the encoded 
data of said variable-length encoding means; 

dummy data generating means for gener- 
ating dummy data which can be eliminated on a 

40 decoding circuit for decoding the encoded data; 

switching means for switching an output of 
said image encoding apparatus to either the en- 
coded data from said temporary storage means 
or to the dummy data from said dummy data gen- 

45 erating means; and 

control means for monitoring a number of 
occupied bits in said temporary storage means 
so as to control a switching operation of said 
switching means so that the output of the image 

50 encoding apparatus becomes the encoded data 

from said temporary storage means when the 
number of occupied bits exceeds a predeter- 
mined threshold, whereas so as to suspend 
transmission of data in said temporary storage 

55 means and control the switching operation of said 

switching means so that the output of the image 
encoding apparatus becomes the dummy data 
generated from said dummy data generating 
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means when the number of occupied bits is not 
more than the threshold. 

2. The image encoding apparatus as defined in 
claim 1, wherein: 5 

said control means outputs a switching 
signal into said encoded data amount control 
means when the number of occupied bits is not 
more than the thresholds; and 

said encoded data amount control means 10 
initializes various parameters which are used for 
calculation of the quantization step size when the 
switching signal is inputted. 

3. The image encoding apparatus as defined in 15 
claim 2, wherein said encoded data amount con- 
trol means includes: 

bit allocating means for calculating a de- 
gree of complexity in an image of a frame to be 
encoded from the encoded results up to the pre- 20 
vious frame so as to determine a number of bits 
to be allocated to the frame to be encoded from 
the degree of complexity; 

reference quantization step size determin- 
ing means for calculating a virtual buffer fullness 25 
based upon a number of generated bits obtained 
from said variable-length encoding means and a 
number of bits allocated by said bit allocating 
means so as to determine a reference quantiza- 
tion step size from the virtual buffer fullness; and 30 

quantization step size determining means 
for calculating activities showing fineness of a 
picture of an image so as to determine a quanti- 
zation step size which is used in the quantizing 
means by changing the reference quantization 35 
step size based upon activities of an image to be 
encoded and an average value of the activities of 
the frame previously encoded, 

wherein the parameters which are initial- 
ized when the switching signal is inputted to said 40 
encoded data amount control means are the de- 
gree of complexity, the initial virtual buffer full- 
ness and the average value of the activities on 
the previous frame. 

45 

4. The image encoding apparatus as defined in 
claim 1, further comprising: 

block dividing means for dividing one 
frame of the inputted image into a plurality of 
macroblocks; and 50 

two-dimensional orthogonal transforming 
means for carrying out two-dimensional orthogo- 
nal transformation on an output of said block div- 
iding means, 

wherein said quantizing means quantizes 55 
an output of said two-dimensional orthogonal 
transforming means. 



5. The image encoding apparatus as defined in 
claim 4, further comprising: 

motion compensation inter-frame predict- 
ing means for while calculating a motion vector of 
each macroblock by comparing a current frame 
with another frame, predicting a macroblock in 
the current frame from another frame, 

wherein intra-frame encoding is per- 
formed on a frame of I picture which appears at 
predetermined frame intervals and inter-frame 
prediction encoding is performed on a frame of 
pictures other than the I picture. 

6. The image encoding apparatus as defined in 
claim 5, further comprising: 

subtracting means for obtaining a differ- 
ence between image data which have been pre- 
dicted by said motion compensation inter-frame 
predicting means and current input image data, 

wherein in the case where inter-frame pre- 
diction encoding is performed, two-dimensional 
orthogonal transformation is carried out on the 
prediction difference obtained in said subtracting 
means by said two-dimensional orthogonal 
transforming means and the transformed predic- 
tion difference is quantized by said quantizing 
means. 

7. The image encoding apparatus as defined in 
claim 5, wherein said motion compensation inter- 
frame predicting means includes: 

inverse quantizing means for inversely 
quantizing the output of said quantizing means; 

inverse orthogonal transforming means 
for carrying out inverse orthogonal transforma- 
tion on the output of said inverse quantizing 
means; 

a frame memory for storing image data 
which have been reconstructed by said inverse 
orthogonal transforming means; and 

motion compensation predicting circuit for 
calculating a motion vector from reconstructed 
image data in said frame memory and current in- 
put image data so as to predict a macroblock of 
a current frame. 

8. The image encoding apparatus as defined in 
claim 1, wherein said dummy data generating 
means generates dummy data with an arbitrary 
data length which include a dummy data start 
code showing a start of the dummy data and a 
dummy data end code showing an end of the 
dummy data. 

9. The image encoding apparatus as defined in 
claim 1, wherein said dummy data generating 
means successively generates predetermined 
dummy codes composed of a plurality of bits. 
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10. The image encoding apparatus as defined in 
claim 1, wherein: 

said variable-length encoding means adds 
a predetermined start code to a front of each 
frame, 

said dummy data generating means gen- 
erates dummy data where same bits continue, 

said control means controls a switching 
operation of the switching means immediately af- 
ter the start code is outputted from said tempor- 
ary storage means so that the dummy data are 
outputted from said dummy data generating 
means successively after generating the start 
code in the case where the output of the image 
encoding apparatus is switched to the dummy 
data. 

11. The image encoding apparatus as defined in 
claim 1, wherein: 

said temporary storage means stores a 
predetermined amount of dummy data which can 
be eliminated on a decoding circuit in an initial 
state, 

the thresholds are set to smaller values 
than the number of occupied bits in said tempor- 
ary storage means in an initial state. 

12. An image encoding apparatus, comprising: 

quantizing means for quantizing informa- 
tion which corresponds to an image; 

variable-length encoding means for per- 
forming variable-length encoding on the quan- 
tized result of said quantizing means; 

temporary storage means for temporarily 
storing encoded data from said variable-length 
encoding means so as to transmit the stored en- 
coded data to an outside at a fixed data rate; and 

encoded data amount control means for 
calculating bit allocation per frame of information 
to be encoded based upon the encoded result of 
said variable-length encoding means, and calcu- 
lating a quantization step size from the above cal- 
culated result, and adjusting the quantization 
step size of quantization in said quantizing 
means according to the above calculated result 
so as to control an amount of the encoded data 
of said variable-length encoding means, 

wherein said encoded data amount control 
means includes bit allocation changing means for 
monitoring a number of occupied bits in the tem- 
porary storage means so as to change the calcu- 
lated bit allocation when the sum of bit allocation 
to a frame which has been calculated before en- 
coding and a number of occupied bits in said tem- 
porary storage means at the time of the calcula- 
tion of the bit allocation exceeds a predetermined 
threshold, such that the sum does not exceed the 
threshold. 



13. The image encoding apparatus as defined in 
claim 12, wherein said encoded data amount con- 
trol means includes: 

bit allocating means for calculating a de- 

5 gree of complexity in an image of a frame to be 

encoded from the encoded results up to the pre- 
vious frame so as to determine a number of bits 
to be allocated to the frame to be encoded from 
the degree of complexity; 

10 said bit allocation changing means for 

changing the bit allocation obtained by said bit al- 
locating means; 

reference quantization step size determin- 
ing means for calculating a virtual buffer fullness 

15 based upon the number of generated bits ob- 

tained from said variable-length encoding means 
and the bit allocation changed by said bit alloca- 
tion changing means so as to determine a refer- 
ence quantization step size from the virtual buf- 

20 fer fullness; and 

quantization step size determining means 
for calculating activities showing fineness of a 
picture of an image so as to determine a quanti- 
zation step size which is used in the quantization 

25 step size determining circuit by changing the ref- 

erence quantization step size based upon activi- 
ties of an image to be encoded and an average 
value of the activities of the frame to be previous- 
ly encoded. 

30 

14. The image encoding apparatus as defined in 
claim 12, wherein said bit allocation changing 
means changes the bit allocation to a value ob- 
tained by subtracting the number of occupied bits 

35 from the threshold when the sum of the bit allo- 

cation to the frame calculated before encoding 
and the number of occupied bits in the temporary 
storage means at the time of calculation of the bit 
allocation exceeds the predetermined threshold. 

40 

15. The image encoding apparatus as defined in 
claim 12, further comprising: 

block dividing means for dividing one 
frame of the inputted image into a plurality of 
45 macroblocks; and 

two-dimensional orthogonal transforming 
means for carrying out two-dimensional orthogo- 
nal transformation on an output of said block div- 
iding means, 

so wherein said quantizing means quantizes 

an output of said two-dimensional orthogonal 
transforming means. 

16. The image encoding apparatus as defined in 
55 claim 1 5, further comprising: 

motion compensation inter-frame predict- 
ing means for while calculating a motion vector of 
each macroblock by comparing a current frame 
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with another frame, predicting a macroblock of old. 
the current frame from another frame, 

wherein intra-frame encoding is per- 
formed on a frame of an I picture which appears 
at predetermined frame intervals and inter-frame 5 
prediction encoding is performed on a frame of 
pictures other than the I picture. 

17. The image encoding apparatus as defined in 

claim 16, further comprising: 10 

subtracting means for obtaining a differ- 
ence between image data which have been pre- 
dicted by said motion compensation inter-frame 
predicting means and current input image data, 

wherein in the case where inter-frame pre- 15 
diction encoding is performed, two-dimensional 
orthogonal transformation is carried out on the 
prediction difference obtained in said subtracting 
means by said two-dimensional orthogonal 
transforming means and the transformed predic- 20 
tion difference is quantized by said quantizing 
means. 



18. The image encoding apparatus as defined in 

claim 16, wherein said motion compensation in- 25 
ter-frame predicting means includes: 

inverse quantizing means for inversely 
quantizing the output of said quantizing means; 

inverse orthogonal transforming means 
for carrying out inverse orthogonal transforma- 30 
tion on the output of said inverse quantizing 
means; 

a frame memory for storing image data 
which have been reconstructed by said inverse 
orthogonal transforming means; and 35 

motion compensation predicting circuit for 
calculating a motion vector from the reconstruct- 
ed image data in said frame memory and current 
input image data so as to predict a macroblock of 
a current frame. 40 



19. The image encoding apparatus as defined in 
claim 1 2, wherein said encoded data amount con- 
trol means further includes quantization step size 
changing means for monitoring a number of occu- 45 
pied bits in said temporary storage means so as 

to change the calculated quantization step size to 
a greater value when the number of occupied bits 
exceeds the predetermined threshold. 

50 

20. An image encoding apparatus in which image 
data is quantized and variable-length encoded 
and then stored into a temporary storage means 
for fixed data rate transmission, and in which 
dummy data which can be eliminated on decod- 55 
ing is output from the apparatus when it is detect- 
ed that the occupied space in the temporary stor- 
age means is less than a predetermined thresh- 
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(54) Image encoding apparatus 

(57) In an image encoding apparatus, image infor- 
mation is quantized and variable-length encoding is per- 
formed thereon so that the data are stored in a buffer. 
Thereafter, the data in the buffer are transmitted to the 
outside at a fixed data rate. At this time, a controller mon- 
itors a number of occupied bits in the buffer, and when 
the number of occupied bits is not more than thresholds, 
the controller switches the output of the apparatus to a 
dummy data generator. At this time, a parameter which 
is used for rate control is initialized, and dummy data 
which can be eliminated on a decoding circuit are out- 
putted from the dummy data generator. Furthermore, bit 
allocation is changed so that a sum of the number of oc- 
cupied bits and the bit allocation to a frame which is de- 
termined before encoding does not exceed .a fixed 
amount. Moreover, when the number of occupied bits ex- 
ceeds the fixed value, a quantization step size is 
changed to a greater value. 



O 




mo 5 h, 



Printed by Jouve (Fn), 18, rue Saint-Denis. 75001 PARIS 



BNSDOCID: <EP 068061 2A3J_> 



EP0 660 612 A3 




Europei 
Office 



latent 



EUROPEAN SEARCH REPORT 



Application N amber 

EP 94 30 9576 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of do 



of relevant 



where appropriate, 



Relevant 
to « 



CLASSIFICATION OF Tiff: 
APPLICATION (IaLCL6) 



US-A-5 235 618 (SAKAI ET AL) 

* column 7, line 42 - column 8, line 47 

* claims 4,5,7 * 

* figures 4,5,9 * 



EP-A-0 536 630 (KABUSHIKI KAISHA TOSHIBA) 

* page 58, line 47 - line 48 * 

* claim 33 * 

* figure 1 * 



SIGNAL PROCESSING IMAGE COMMUNICATION. , 
vol. 5, no. 1/2, February 1993 AMSTERDAM 
NL, 

pages 21-37, XP 000345611 

CHEN ET AL 'Hybrid extended MPEG video 

coding algorithm for general video 

applications 1 

* page 28, left column, paragraph 3 - page 
29, right column, paragraph 1 * 

* figure 3 * 



VISUAL COMMUNICATION AND IMAGE PROCESSING 
'91: VISUAL COMMUNICATION, 
vol. 1605, no. 1, 11 November 1991 BOSTEN, 
MA, US, 

pages 58-72, XP 000479218 

VISCIT0 ET AL 'A VIDEO COMPRESSION 

ALGORITHM WITH ADAPTIVE BIT ALLOCATION AND 

QUANTIZATION 1 

* page 62, last paragraph - page 65, 
paragraph 1 * 

* page 68, last paragraph * 

* page 71, paragraph 2 * 

* figure 2 * 

-/— 



1,20 

3-8,10 
20 

2-7 

12,14-18 



H04N7/24 



TECHNICAL FIELDS 
SEARCHED (InLCLo) 



13,19 
12-19 



H04N 



The present search report has been drawn up for all dm 



Plan of March 



THE HAGUE 



Date mi oanpfcitM *f the utfc 

17 January 1996 



Dippel, U 



CATEGORY OF CITED DOCUMENTS 

X : particularly relevant if taken aloae 

Y : particularly relevant if combined with another 

document of the saa 
A : technological background 
O : non-written disclosure 
P : intermediate docaaaent 



T : theory or principle n adi Hy ing the invention 
E : earlier patent docnaent, hot aahlshed on, or 

after the fling date 
D : document cited in the application 
L : <<imnmt died for ether reasons 

& : member of the same patent family, correspondine 



2 



BNSDOCID: <EP 066061 2A3 I > 



EP 0 660 612 A3 



3 



European Patent 
Office 



CLAIMS INCURRING FEES 



The present European patent application comprised at the erne of filing more than en darns. 

□ AH claims tees have been paid vwthin the prescribed time limit The present European search report has been 
drawn up tor all daims. 

Q Only part of the daws tees have been paid within the prescribed time emit The present European search 
report has been drawn up tar the first ten daims and for those daims tor which daims tees have been paid. 

namely daims: 

□ No claims tees have been paid within the prescribed time limit The present European search report has been 
drawn up tor the first ten daims. 



LACK OF UNITY OF INVENTION 



The Search Division considers that the present European potent 
r> vennon and relates to several inventions or groups of inventions, 
namely: 



see sheet -B- 



n do 



s not comply wrth the requirement of unity of 



r-j All further search lees have been paid withm The fixed time limit. The present European search report has 
been drawn up tor all daims. 

□ ° n, y P 3 " 0,the furmer search tees have been paid within the fixed time irnit. The present European search 
report has been drawn up tor those parts of the European patent ap plica ton which relate to the invensons w> 
respects of which search tees have been paid. 

namely claims: 



□ None of *» k«*er search tees has been paid within the faced time Irnit The present European 
has been drawn up for those parts of the European patent application which relate to the 
mentioned in the daims. 

namely daims: 



search report 
first 



BNSDOCID: <EP_ 



_ 066061 2A3_I_> 



EP 0 660 612 A3 



European Patent 
Office 



EUROPEAN SEARCH REPORT 



Application Number 

EP 94 30 9576 



DOCUMENTS CONSIDERED TO BE RELEVANT 



Category 



Citation of document with indication, where appropriate, 
of relevant passages 



Relevant 
to I 



CLASSIFICATION OF THE 
APPLICATION (InLCU) 



P. A 
A 



P. A 
A 

A 
A 



US-A-5 317 397 (ODAKA ET AL) 31 May 1994 
& PATENT ABSTRACTS OF JAPAN 
vol. 17 no. 470 (E-1422) ,26 August 1993 
& JP-A-05 111012 (TOSHIBA CORPORATION) 
30 April 1993, 

* column 22, line 56 - column 23, last 
paragraph * 

* column 26, line 29 - line 36 * 

* figure 32 * 

US-A-4 989 130 (M0RIYAMA ET AL) 

* column 9, line 35 - line 68 * 

* column 11, line 26 - line 37 * 

* figure 4 5 6B 9 * 

US-A-5 319 457 (NAKAHASHI ET AL) 7 June 
1994 

& JP-A-05 068 243 (HITACHI LTD) 19 March 
1993 

* column 4, line 15 - line 29 * 

* column 5, line 1 - line 7 * 

US-A-5 163 056 (HAG I WAR A ET AL) 

* figures * 

EP-A-0 493 136 (VICTOR COMPANY OF JAPAN 
LTD) 

* column 5, line 44 - column 6, line 36 * 

* figure 1 * 

* column 10, line 29 - column 11, line 16 
* 

* claims 1,2 * 

US-A-5 038 209 (HANG) 

* column 4, line 3 - line 34 * 

* claims 6-8,12,13 * 

* figure 3 * 



The present search report has been drawn up for all 



1-7 
1-7 



1,2,8-11 

1,3-7 
1,3-7 



TECHNICAL FIELDS 
SEARCHED <1dLCL6) 



2,8-11 



12,13, 
15-19 



13,19 



Place of Match 



THE HAGUE 



17 January 1996 



Dippel, U 



CATEGORY OF CITED DOCUMENTS 



i 

Z 

c 

m. 

C 



X : particularly r*i< 
Y 



A: 

O : nw>- written 4isd 
P : intermediate document 



T : theory or principle uatertyiag the invention 
E : earlier patent toco neat, but published on, w 

after the riling date 
D : document cited in the application 
L : docaaaent cited tor other reasons 

aber of the same patent family, 



4 



BNSDOCID: <EP 066061 2A3_I_> 



EP0 660 612 A3 



European Patent 
Office 



EP 94 30 9576 -B- 



LACK OF UNITY OF INVENTION 



Tho G — U i Owwoni 
Invsnson ond rolws 
n*m«ty: 



ttwi +m pr— Q1 Eutpp— n pfftt i 
( trwanttons or Qnaups of inventions. 



Ojmm not oofnpty witfi £m rvquramwtt oi unity of 



1. Claims 1-11,20: In order to avoid underflow of the buffer 

used to adopt a variable data rate to a 
channel with fixed data rate, the encoder 
is completely disconnected from the channel, 
instead, dummy data are sent to the channel 
with the data rate required by the channel 

2. Claims 12-19 : In order to avoid the overflow of the said 

buffer, the quantization step size is changed 
in order to generate a smaller amount of data 



THIS PAGE BLANK 



(USPTO) 



